Introduce parser_errors spec test suite#137
Merged
Merged
Conversation
54890e5 to
165af5c
Compare
The error matcher (adapter_runner.rb#matches_patterns?) does case-insensitive substring/regex matching against both the full error text and the extracted core message. Patterns only need to capture the semantic essence of the error. The recorded patterns embedded the input template verbatim, e.g. 'Liquid syntax error: Expected end_of_string but found close_round in "x = x )"' That overfits to liquid-ruby's exact wording AND to the exact template text, which means any implementation with slightly different quoting/formatting fails. Trim 1901 patterns down to their semantic core: 'Expected end_of_string but found close_round' This matches the existing convention in specs/liquid_ruby/strict_rejects_lax_syntax.yml (short substrings like 'but found pipe', 'Expected close_square'). Result: 80 unique patterns, max length 95 chars (was 1901 unique, often >100). Verified against liquid_ruby adapter: 1791/1901 specs match correctly; the 110 remaining failures are specs whose enforcement requires strict2_parsing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
liquid-spec doesn't have many cases covering syntax errors. This PR adds ~2K specs covering nearly every
SyntaxErrorbranch across all tags — all additive YAML, no Ruby changes.Each spec follows this shape:
How adapters opt in
The suite is opt-in, so existing adapters see nothing new. An adapter that supports strict2 parsing enables the full suite by adding it to their suite list; the existing feature-gating system handles the rest.
Coverage
The specs aim to cover every
raise SyntaxErrorcall site in the parser including unguardedparser.consume(type)calls.